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PROTECTING IMAGES WITH AN IMAGE WATERMARK 



2 FIELD OF THE INVENTION 

3 This application relates to the field of digital imaging. It is 

4 more specifically concerned with the insertion and detection of 

5 an identifying mark on a work-piece. 



BACKGROUND OF THE INVENTION 

(Rj It is a constant endeavor to find improved techniques of placing 

jiy a visible or invisible identifying mark on an image. This is 

¥ % generally useful to establish ownership, origin and authenticity, 

l(T and also to discourage those who might wish to purloin or 

lP misappropriate the work. Identifying marks are also useful to 

lps, give evidence of unauthorized alteration or disclosure. 

o 

lpi. Visible marks are herein classified as being either visible 

14 robust or visible fragile. A mark is classified as visible robust 

15 if it can be seen by the unaided eye and cannot be easily removed 

16 from the work-piece, if at all, without leaving telltale 

17 evidence. It is classified as visible fragile if the mark itself 

18 is visibly altered by an attempt to alter the work-piece or its 

19 wrapper. 

20 Invisible marks are herein classified relative to the appearance 

21 of that mark to a human being with normal visual acuity. A mark 

22 on an image is classified as having an invisibility 
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1 classification level of undetectably invisible if, when the image 

2 without the marking is displayed together with an image copy with 

3 the marking, the human being is equally likely to select either 

4 of these copies. An undetectably invisible mark is below or at 

5 the human being's just noticeable difference. A mark on an image 

6 is classified as having an invisibility classification level of 

7 subliminally invisible if the mark is not distracting to the 

8 human viewer, although it is above the human being's just 

9 noticeable difference. An image mark is classified as being 
10 marginally invisible if it does not cause the marked image to 
1W lose its usefulness or value because of the mark. An image 

12g marking is classified as being poorly invisible if the marking 

l|Jf causes a reduction in the usefulness and/or value of the image. 
Hi 

l&j Presently, both visible and invisible markings of hardcopy 

1§ documents are used as a generally dependable method of 

lfej establishing ownership and authenticity. These time-tested 

1^ methods are also useful for marking a "softcopy" digitized image, 
m 

1§2 also referred to herein as an image. A digitized image is an 

1#* abstraction of a physical image that has been scanned and stored 

20 in a computer's memory as rectangular arrays of numbers 

21 corresponding to that image's (one or more) color planes. Each 

22 array element corresponding to a very small area of the physical 

23 image is called a picture element, or pixel. The numeric value 

24 associated with each pixel for a monochrome image represents the 

25 magnitude of its average brightness on its single color (black 

26 and white) plane. For a color image, each pixel has values 

27 associated and representing the magnitude or average brightness 

28 of its tristimulus color components representing its three color 

29 planes. Other image representations have more than three color 
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1 components for each pixel. A different component value is 

2 associated with each different one of the image's color planes. 

3 In what follows, whenever reference is made to color planes it is 

4 understood to include any number of color planes used by a 

5 particular image's digitizing technique to define the pixel's 

6 color characteristics. This includes the case when there is only 

7 a single plane defining a monochrome image. 



8^ A digitized image is recognizable as an image to a human viewer 

^ only when the individual pixels are displayed as dots of white or 
colored light on a display or as dots of black or colored inks or 

IN* dyes on a hardcopy. Pixels are normally spaced so closely as to 

ljU be unresolvable by the human visual system. This results in the 

13 fusion of neighboring pixels by the human visual system into a 

l|g representation of the original physical image. Image fusion by 

110 the human visual system makes invisible marking, or relatively 

l|Jj invisible marking, of images possible. This property is fully 

lip exploited by the methods described here to both impart upon a 

U 

18 digitized image an invisible watermark to a desired invisibility 

19 classification, and to subsequently demonstrate its existence. 

20 The imparting and demonstrated detection of a robust invisible 

21 marking on digitized images, herein called invisible 

22 watermarking, are a primary aspect of the present invention. 



23 PROPERTIES OF A ROBUST INVISIBLE WATERMARK 

24 A proper invisible watermarking technique that imparts an 

25 invisible watermark upon a proprietary digitized image should 

26 satisfy several properties. The imparted watermark should appear 
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1 to be invisible to any person having normal or corrected visual 

2 accommodation to a desired invisibility classification level. 

3 Clearly, the degree of marking is a dichotomy. A balance has to 

4 be struck between protecting the image from unauthorized uses and 

5 not having the watermark unpleasantly alter the appearance of the 

6 image. This generally means that a recognizable pattern should 

7 not appear in the marked image when the watermark is applied to a 

8 uniform color plane. This, requirement discourages marking the 

9 image by varying the hue of its pixels, since the human visual 

0 system is significantly more sensitive to alterations in hue than 

|sj in brightness. The requirement can be satisfied by a technique 

fij based on varying pixel brightness implemented in a proper way. A 
S3 

gj technique based on varying pixel brightness also allows the same 

¥r marking technique applied to color images to be equally 

j|j applicable to monochrome images. 

© 

a 

j*j Another property of a proper invisible watermarking technique is 

68 that it should have a detection scheme such that the probability 

^ of a false-positive detection is vanishingly small. For purposes 

19^ of the present invention, the probability of detection of a 

20 watermark in an image when one does not exist should be less than 

21 one in a million. There is generally little difficulty satisfying 

22 this requirement when the technique is statistically based. 

23 Still another property of a proper watermarking technique is that 

24 it should be possible to vary the degree of marking applied to an 

25 image. In this way, the watermark can be made as detectable as 

26 necessary by the particular application. This property is 

27 important in highly textured images where it is often necessary 

28 to increase the intensity of the mark to increase its likelihood 

29 of detection. This is in contradistinction with images that have 
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1 low contrast in which it is advantageous to reduce the marking 

2 intensity to lessen undesirable visible artifacts of the 

3 watermark itself. 

4 It is also highly desirable that when detected the demonstrated 

5 existence of the watermark should be translatable to a 

6 recognizable visual image having relatively bold features with a 

7 high contrast ratio. Features of a demonstrated visual image that 

8 are not relatively bold may otherwise be difficult to show if the 

9 watermark has been attacked in attempts to defeat its protection. 
□ 

1^ Finally, the imparted watermark should be robust in that it 

iy should be very difficult to be removed or rendered undetectable. 

l|y It should survive such image manipulations that in themselves do 

IS* not damage the image beyond usability. This includes, but is not 

If limited to, JPEG "lossy" compression, image rotation, linear or 

lip nonlinear resizing, brightening, sharpening, "despeckling," pixel 

m 

1|2 editing, and the superposition of a correlated or uncorrelated 

1¥* noise field upon the image. Attempts to defeat or remove the 

l|E} watermark should be generally more laborious and costly than 

19 purchasing rights to use the image. If the image is of rare 

20 value, it is desirable that the watermark be so difficult to 

21 remove that telltale traces of it can almost always be recovered. 

22 SUMMARY OF THE INVENTION 

23 An aspect of the present invention is to provide a method for 

24 imparting a watermark onto a digitized image comprising the steps 

25 of providing the digitized image, and multiplying the brightness 

26 data associated with at least one of the image pixels by a 

27 predetermined brightness multiplying factor. The image includes a 
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plurality of pixels, wherein each of the pixels includes 
brightness data that represents one brightness value if the image 
is monochrome, or a plurality of brightness data values if the 
image has multiple colors. A brightness data value of a pixel and 
a color component or . component are hereinafter used to mean the 
same thing, and are therefore to be considered interchangeable. 
In an embodiment, the brightness multiplying factor ranges from 
0.5 to 1.0. Other smaller or larger factors are useful in some 
image applications dependent upon particular desired watermarking 
results. The brightness multiplying factor has a relationship 
with a number taken from a random number sequence and the 
relationship is a linear remapping to provide a desired 
modulation strength. 

In an embodiment, each of the pixels has a row and a column 
location in an array representing the digitized image, and the 
brightness multiplying factor employs a different sequential 
combination of numbers from a robust random number sequence in 
sequential correspondence to the row and column location. 

19 Another aspect of the present invention is to provide a method 

20 for generating a watermarked image wherein a watermark is 

21 imparted onto a digitized image having a plurality of original 

22 pixels, each pixel having original brightness values. The method 

23 includes the step of providing a digitized watermarking plane 

24 comprising a plurality of watermarking elements, each having a 

25 brightness multiplying factor and having one-to-one positional 

26 correspondence with the original pixels. It also includes the 

27 step of producing a watermarked image by multiplying the original 

28 brightness values of each of the original pixels by the 

29 brightness multiplying factor of a corresponding one of the 
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1 watermarking elements wherein the watermark is invisible. In an 

2 embodiment/ when the original image forms an original plane and 

3 the watermarking plane is smaller than the original plane, the 

4 method further includes the step of extending the watermarking 

5 plane by tiling such that the watermarking plane covers the 

6 original plane and/or further comprises the step of truncating 

7 the watermarking plane such that the watermarking plane covers 

8 the original plane, upon determining that the watermarking plane 

9 extends beyond the original plane. 

lfih Another aspect of the present invention is to provide a method 

110 for forming a watermarking plane including a plurality of 

ljTj elements each having a multiplying value. The method comprises 

the steps of: generating a robust random sequence of integers 

lflg, having a first plurality of bits; linearly remapping the random 

15* sequence to form a remapped sequence of brightness multiplying 

16g factors to provide a desired modulation strength; computing a 

if? discrete Fourier transform of the remapped sequence to form a 

lgl Fourier sequence having frequency coordinates; expanding the 

lP frequency coordinates to form an expanded sequence; and computing 

20 an inverse Fourier transform of the expanded sequence to obtain a 

21 watermarking sequence of values. 

22 An embodiment further includes one or more or the following: the 

23 step of expanding is accomplished by zero-padding; the method 

24 further comprises a step of employing the watermarking sequence 

25 to provide the multiplying value for each of the elements; the 

26 method further comprises the steps of hard clipping the 

27 watermarking sequence to form a hard-clipped sequence having 

28 sequence members, and utilizing a different one of the sequence 

29 members to provide the multiplying value for each of the 
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1 elements; the method further comprises the steps of adjusting the 

2 watermarking sequence to form a normalized sequence of values 

3 having a mean and a median equal to the difference between unity 

4 and the modulation strength, and having a maximum of unity, and 

5 employing the normalized sequence to provide the multiplying 

6 value for each of the elements; the method further comprises the 

7 steps of providing an unmarked original image having a plurality 

8 of original pixels, each of the pixels having at least one 

9 component, wherein a first number of the original pixels is 
1% greater than a second number of the plurality of elements, 

IHP expanding the watermarking plane by tiling to cover the unmarked 

l£y original image such that one of each of the pixels has one 

1** corresponding element from the elements; and multiplying the at 

14& least one component of each of the pixels by the multiplying 

IS! value of the corresponding element. 

9 

l|3 Still another aspect of the present invention is to provide a 

1|F] method for detecting a watermark in a marked image. The marked 

18^ image is marked by a watermarking plane which has a plurality of 

19 watermarking elements. Each of the image pixels has at least one 

20 component and each of the watermarking elements has a brightness 

21 multiplying factor. The method employs a selector having at least 

22 one element and a visualizer having at least one pixel and at 

23 least one counter, said at least one counter to store the 

24 comparison data resulting from comparisons for each of a 

25 plurality of selector elements and positions; said comparison 

26 data resulting from the comparison of the statistical brightness 

27 of each image color component, relative to its neighboring color 

28 components in the same plane, with the statistical magnitude of 

29 each corresponding brightness multiplying factor, relative to its 

30 neighboring multiplying factors. The method further comprises the 
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1 step of displaying a visualizer-coincidence image such that a 

2 user can make a determination as to whether the pattern encoded 

3 by the visualizer pixels is recognizable and thereby, whether the 

4 watermark is detected. 

5 Further, it is an aspect of the present invention to provide an 

6 alternative method and apparatus for imparting a watermark into 

7 a digitized image that includes the step of providing the 

8 digitized image and the step of adding at least one predetermined 

9 brightness adjusting value to the brightness data associated with 

at least one of the image pixels. The image includes a plurality 

110 of pixels, wherein each of the pixels includes brightness data 
UJ 

12j, that represents one component if the image is monochrome, or a 

lR* plurality of components if the image has multiple colors. The 

1^ step of "adding a predetermined brightness data value to a 

15. component" is used in the same way as the step of "multiplying a 

l|g component by a predetermined brightness multiplying factor", 

l¥> where the component is associated with at least one of the image 

lj||. pixels. Under conditions that will be specified, the step of 

1^ adding achieves image watermarking results which are similar in 

20 every manner and respect to the step of multiplying. The additive 

21 brightness adjusting values may be positive or negative, and a 

22 color component altered by the step of adding increase or 

23 decrease accordingly. 

24 In another aspect of a general embodiment, the components of all 

25 image pixels, or all image pixels in a specified image portion, 

26 are each modified by an associated brightness adjusting factor. 

27 In another particular embodiment, each of the pixels has a row 

28 and a column location in an array representing the digitized 
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1 image, and the brightness adjusting factors for each pixel employ 

2 a different sequential combination of numbers from a different 

3 robust random number sequence in sequential correspondence to the 

4 row and column location. 

Another aspect of the present invention is to provide a method 
for generating a watermarked image . wherein watermarks are 
imparted into a digitized image by having a plurality of original 
watermarking elements, with each of the elements having an 
original brightness adjusting value. 

Another aspect of the present invention is to provide a method 
for forming watermarking planes. Each watermarking plane includes 
a plurality of elements with at least one brightness adjusting 
value derived from each element. 

Still another aspect of the present invention is to provide a 
method for improving the probability of detection of a watermark 
in a marked image or a derived copy of a marked image. The marked 
image is marked by a watermarking plane which has a plurality of 
watermarking elements. The method applies a two-dimensional 
blurring filter to the marked image or derived copy of a marked 
image prior to attempted detection of the imparted watermark. 

21 BRIEF DESCRIPTION OF THE DRAWINGS 

22 These and other objects, features, and advantages of the present 

23 invention will become apparent upon further consideration of the 

24 following detailed description of the invention when read in 

25 conjunction with the drawing figures, in which: 
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1 FIG. 1 shows a block diagram of an image capture and distribution 

2 system suitable for use in accordance with an embodiment of the 

3 present invention. 

4 FIG. 2 shows an embodiment for forming a watermarking plane in 

5 accordance with the present invention. 

6 FIG. 3 shows an embodiment for the steps of watermark imparting. 

13 FIG. 4 shows an overview of the steps for image alignment. 

^ FIG. 5 shows the steps for a coarse alignment of a marked image 

$j with a correlation reference plane. 

H» 

1Q FIG. 6 shows the steps for a fine alignment of a marked image 

1.W with a correlation reference plane. 
00 

H 

1|H FIG. 7 shows the steps for finding a watermark in a marked image. 

13 FIG. 8 shows a random positioning of the selector array over the 

14 watermarking plane and the image planes. 

15 FIG. 9 shows a typical visualizer pattern. 
16 

17 FIG. 10 shows a method of verification of the presence of the 

18 watermark. 

19 FIG. 11 shows a detection resulting from the visualizer of Figure 

20 9 for a watermarking made at a modulation strength of 1%. 
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1 FIG. 12 shows a detection resulting from the visualizer of Figure 

2 9 for a watermarking made at a modulation strength of 2%. 

3 FIG. 13 shows a detection resulting from the visualizer of Figure 

4 9 for a watermarking made at a modulation strength of 4%. 

5 FIG. 14 shows a detection resulting when the image has no 

6 watermark. 

7 FIG. 15 shows the steps for an alternate method of finding a 
^ watermark in a marked image. 

|)y FIG. 16 shows an enlarged segment of a watermarked image, having 

iff been watermarked at a modulation strength of 2.5%, that is used 

FU 

1|U as the reference image. 
D 

lfg FIG. 17 shows the enlarged segment of the reference image after 

ly* it has been prepared for printing by screening, has been printed 

ltfl and has been scanned to form the derivative image. 

6 

15 FIG. 18 shows the enlarged segment of the derivative image that 

16 has been acted upon by a blurring filter to form the filtered 

17 image. 

18 FIG. 19 shows a visualizer-coincidence image resulting from a 

19 watermark detection made on the reference image. 

20 FIG. 20 shows a visualizer-coincidence image resulting from a 

21 watermark detection made on the derivative image. 
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1 FIG. 21 shows a visualizer-coincidence image resulting from a 

2 watermark detection made on the filtered image. 

3 DETAILED DESCRIPTION OF THE INVENTION 

4 The present invention provides a robust means of watermarking a 

5 digitized image with a highly random sequence of pixel brightness 

6 multipliers. The random sequence is formed from four 

7 ^robust-watermarking-parameters' selected and known only by the 

8 marker and/or the marking entity. A watermarking plane is 

9% generated which has an element array with one-to-one element 
correspondence to the color component array or arrays of the 

l|y digitized image being marked. Each element of the watermarking 

Mb 

l§y plane is assigned a random value dependent upon a robust random 

15* sequence and a specified brightness modulation strength. The so 

\%r generated watermarking plane is imparted onto the digitized image 

1§3 by multiplying the brightness value or values of each pixel by 

its corresponding element value in the watermarking plane. The 

IIP resulting modified brightness values impart the random and 

relatively invisible watermark onto the digitized image. 

19 Detection of an imparted watermark requires knowing the 

20 watermarking plane with which the watermark was imparted. 

21 Regeneration of the watermarking plane requires knowledge of the 

22 robust-marking-parameters used in its formulation. This is 

23 generally only known to the marker and/or marking entity. Once 

24 regenerated the watermarking plane is used together with a 

25 verifying image located in a x visualizer' to demonstrate the 

26 existence of the watermark. 

27 Brightness modulation is the essence of watermark imparting 

28 according to the present invention. Pixel brightness, as used 
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1 herein, expresses the brightness of a visual stimulus in terms of 

2 the CIE 1931 Standard Colorimetric Observer and Coordinate System 

3 tristimulus component brightness X, Y and Z that correspond to a 

4 matching mixture of three reference stimuli. If the image is 

5 monochrome, pixel brightness expresses the brightness of a visual 

6 stimulus in terms of the CIE 1931 Standard Coordinate System 

7 photopic brightness Y, and components X and Z have no meaning. A 

8 more detailed description of pixel brightness is found in G. 

9 Wyszecki and W. S. Styles, "Color Science: Concepts and Methods, 
10 Quantitative Data and Formulae," John Wiley & Sons, Inc. (2nd 
1L ed.), New York, 1982, pp. 164-169, incorporated herein by 

123 reference in its entirety. The CIE 1931 standard specifies three 

ljj particular reference stimuli. The stimuli are radiometric 

Ijfe quantities, and as such are expressed in radiometric units such 

as watts. Grassmann's law, on which nearly all of modern 

1@ colorimetry is based, requires use of the three specific 

\% reference stimuli, or three others that are distinct linear 

1 §Q combinations of them. This is discussed in D. B. Judd and G. 

1^ Wyszecki, "Color in Business, Science, and Industry," (3rd ed.), 

2Q8 John Wiley & Sons, Inc., New York, 1975, pp. 45-47, incorporated 

21 herein by reference in its entirety. By modifying only a pixel's 

22 brightness, its color, represented by its hue and saturation, is 

23 not changed. This is accomplished by preserving the ratios of X:Y 

24 and Z:Y while changing the magnitude of Y. A pixel represented in 

25 any nonlinear color space, such as the color space of the 

26 subtractive dyes Cyan, Magenta, Yellow and Black (CMYK) used in 

27 color printing, will be translated to the X,Y,Z color space (or 

28 to a color space linearly related to it) before the pixel's 

29 brightness is modified. 
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1 Figure 1 shows a block diagram of a system embodiment for 

2 imparting a relatively invisible watermark on a digitized image 

3 in accordance with the present invention. Figure 1 shows an image 

4 capture and distribution system 150 suitable for use in 

5 accordance with an embodiment of the present invention. A scanner 

6 100 captures image data 101 from a physical source 102. The 

7 physical source 102 is. typically a painting or photograph. The 

8 image sends data 101 to a digital computer 104. The computer 104 

9 includes a working storage 106 that is typically embodied in the 
10 computer's random access memory, an image storage system 108 that 
1Q is often a conventional hard disk drive, and an image archive 110 

that can be a tape or disk storage. The computer 104 also 

lSy includes a number of software modules. These include front end 

image processing software 112 that performs image processing such 

liH* as scaling and enhancement of the image data provided by the 

1(T scanner 100. It also includes color preserving watermarking 

lf3 software 114 operating in accordance with the principles of the 

1^ present invention, and back-end image processing software 116 

ISP that performs other processing functions such as compression on 

20^ the watermarked image. Most often, the unprocessed or front-end 

21 digitized original image 101 is sent to the image archive 110 for 

22 preservation in unwatermarked form. 

23 An alternate embodiment has the original image already available 

24 in digitized form 101 without requiring a scanner 100. The 

25 watermarking software 114 applies a relatively invisible 

26 watermark to the digitized image 101 in accordance with the 

27 principles of the present invention. The watermarking process can 

28 also be performed on a copy of an archived image or on other 

29 scanned and processed image data, which has been loaded in whole 

30 or in part, into the computer's working storage 106. 
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1 The processed, watermarked and compressed image produced by the 

2 combination of the software modules 112-116 is sent from the 

3 working storage 106 or image storage 108 to an image server 118 

4 that is connected to a digital network 120. When appropriate, the 

5 digital network is interconnected with a Local Area Network 

6 (LAN), a Wide Area Network (WAN) such as the Internet, or both. 

7 Other systems 122 connected to the digital network 120 can 

8 request and receive images stored on the image server 118 via the 

9 digital network 120. In some cases, the systems can then display 
€p the received images on a display device 124 and/or print the 
|| images on a graphics capable printer 126. Those skilled in the 
M art will recognize that there are many other system 
3y configurations in which the present invention could be employed. 
It* The system of Figure 1 is generally also useful for detecting and 
fT demonstrating the existence of the watermark in a manner such as 
p those described subsequently. 

P MARKING AN IMAGE WITH A ROBUST WATERMARK 

□ 

19 In one embodiment, the watermark imparted onto the digitized 

20 image is a monochrome pattern, herein called "the watermarking 

21 plane/' that overlays the digitized image. The pattern is 

22 embodied by selecting its element values from a robust random 

23 sequence formed from a group of robust sequence generating 

24 parameters. The parameters are used to generate a generally 

25 strongly encrypted random sequence in a manner well known to 

26 those skilled in the art. These parameters are herein referred to 

27 as the ^robust-watermarking-parameters' . In a preferred 

28 embodiment, these parameters include a cryptographic key, two 
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1 coefficients of a linear random number generator, and an initial 

2 value of the random number generator. 

3 Each value, or group of values, of the robust random sequence is 

4 associated with one of the pixels of the digitized image. Most 

5 often the values of the random sequence are linearly remapped to 

6 meet particular criteria. All the brightness values of the 

7 .plurality of color planes of each pixel are multiplied by its 

8 associated linearly remapped robust random sequence value called 
|g its brightness multiplying factor or multiplying factor. A 

brightness multiplying factor which modifies pixel brightness 
values by less than ten percent is herein referred to as a 

&4 brightness multiplying factor producing a relatively invisible 

|L watermark. It is noted that depending on the texture of the image 

43 being watermarked, the brightness values are generally modified 

|a on average by a percentage factor of only 0.3 to 4 percent, and 

§3 rarely up to 10 percent. This is in order to make the marking 

Mb 

^ less visible. The percentage factor is herein referred to as the 

P modulation strength. The actual modulation strength employed is 

9 dependent upon the classification level of invisibility required 

20 by the particular use. It is not advisable to employ a brightness 

21 multiplying factor greater than .unity. This can result in some 

22 pixel brightness values being greater than one. If employed, it 

23 is recommended that all brightness values greater than one be 

24 clipped to a value of unity. This can alter the pixel's color, 

25 thus altering the appearance of the image. 

26 Imparting a watermark upon a digitized image by varying the 

27 brightness of each pixel with a multiplying factor maintains each 

28 pixel's color by satisfying Grassmann's law. A compromise is 

29 generally made in selecting modulation strength. A smaller 
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1 percentage makes the watermark less visible but more difficult to 

2 detect. A larger percentage makes it easier to detect and 

3 demonstrate the existence of the watermark, but makes the 

4 watermark more visible. A highly textured image may require the 

5 use of a relatively high modulation strength. An imparted 

6 watermark is considered to be undetectably invisible in all cases 

7 if the modulation strength is less than 0.5 percent, even when 

8 the unmarked digitized image is a uniform medium gray. For 

9 digitized images having more practical . and valuable features, 
10 subliminally invisible watermarks generally have modulation 
1^ strengths of 1% to 3%, depending on the degree of textural 
13P variation in the image itself. 

w 

lb^ . The watermark imparted in accordance with this invention is 

1^ selected so as to appear to be relatively invisible to any person 

lfP having normal or corrected visual accommodation. The probability 

l|p of a false-positive detection of the watermark in an image when 

it does not exist is less than one in a million. It is possible 

18] to vary the degree of imparted watermarking onto the image so 

that the watermark can be made as detectable as necessary 

20 consistent with a required invisibility classification. The 

21 detected watermark is translatable to a recognizable visual 

22 image, called a visualizer, having relatively bold features and 

23 with a very high contrast ratio. The watermark once imparted, is 

24 very difficult to remove or to be rendered undetectable without 

25 reducing the usefulness and/or value of the digitized image. 

26 In an embodiment of this invention, marking a digitized image 

27 with an invisible watermark requires the formation of a plane for 

28 watermarking. The invisible watermark is herein represented as a 

29 rectangular array of numeric elements, henceforth referred to as 
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the watermarking plane, having I rows and J columns. The I rows 
and J columns correspond to the dimensions of the entire original 
digitized image, or a portion thereof, to which it is being 
applied. 

When an original digitized image is very large, a generated 
watermarking plane not large enough to cover the entire original 
image is extended by tiling replication in any direction 
necessary to cover the entire image. If a watermarking plane 
being so tiled extends beyond any edge of the original image, the 
watermarking plane is assumed to be truncated. These conventions 
are adopted for this embodiment to allow every pixel of the 
original image to have its brightness altered and to ensure that 
the marked image is equal in size to the original image. This 
forms a one-to-one correspondence between element locations in 
the watermarking plane and color components in the color planes 
of the original image. This is generally a desirable 
implementation, even though alternate embodiments do not require 
watermarking the entire image. 

In a preferred embodiment, the value of each element in the array 
defining the watermarking plane is linearly remapped to be a 
random number in the range, 



1 > w(i,j) > (1 - 2p), 



(1) 



where, 



1 < i < I, and 



(2) 



1 < j < Jr 



(3) 
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are the row and column indices of the array, and P is the 
modulation strength of the watermark such that, 



0.5 > p > 0. 



(4) 



Additionally, all elements in the generated watermarking plane, 
treated as an ensemble, are adjusted to have a mean and median of 



Imparting the watermark onto an image begins with generation of 
this watermarking plane. The watermark is imparted onto the 
original image by multiplying all the brightness values 
associated with every pixel in each color plane by the value in 
its corresponding element in the watermarking plane. 

CONSTRUCTING THE WATERMARKING PLANE 

The construction of the watermarking plane is fundamental to 
insuring the robustness of the imparted watermark and its ability 
to survive even determined attacks. To this end, the procedure by 
which the values of the watermarking plane elements are chosen is 
based on cryptographic and two-dimensional signal processing 
theory techniques. These are used to satisfy particular requisite 
properties of the watermarking plane. 

The Property of Unpredictable Randomness 

Consideration is now given to the values of the watermarking 
plane elements to meet the property of unpredictable randomness. 
Unpredictable randomness requires that each element's value 



i-p. 
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1 should vary randomly from the values of its neighbors, and the 

2 seguence of element values should be essentially unpredictable. 

3 Random variation of the elements is reguired for the watermark to 

4 be rendered relatively invisible. In as much as pattern 

5 recognition is one of the most dominant characteristics of the 

6 human visual system, the presence of any pattern among elements 

7 of the watermarking plane could make it visible. The 

8 unpredictability of the seguence of values is reguired to make 

9 the watermark robust and less vulnerable to attack. It is 

10 recognized that if all values in the watermarking plane could be 

l|g predicted, the watermarking process could easily be reversed and 

lS? the mark removed. This could thereby be used to essentially 

1 §y restore the marked image to a nearly perfect copy of the original 

l|j unmarked image. Thus, a means of generating a highly 
unpredictable random number seguence is preferred. 

□ 

1® Generating random values by a congruence method, typical of 

lf-J nearly all popular pseudo-random number generating algorithms, is 

lgTl not considered herein to provide an adeguate level of 

l|? unpredictability. These seguences have only modest cryptographic 

20 strength and are relatively easily discernible by crypto-analytic 

21 techniques. This is described in "The Cryptographic Security of 

22 Truncated Linearly Related Variables," J. Hastad and A. Shamir, 

23 Proceedings of the 17th Annual ACM Symposium on the Theory of 

24 Computing, 1985, pp 356-. 362, which is herein incorporated by 

25 reference. 

26 For the purposes of this invention a sequence is generated by 

27 using a strong cryptographic method such as the National Standard 

28 Data Encryption Algorithm. This is described in: "American 

29 National Standard Data Encryption Algorithm/' ANSI X3. 92-. 1981, 
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1 American National Standards Institute, New York; and in A. G. 

2 Konheim, et al., "The IPS Cryptographic Programs/' IBM System 

3 Journal, Vol. 19, No. 2, 1980, pp 253-283; which are herein 

4 incorporated by reference. 

5 The data sequence of eight-bit values to be encrypted is selected 

6 by the marker, and is desirably generated by a congruence 

7 algorithm. However, the robust secure sequence is produced by 

8 action of the strong encryption algorithm on that data. Using 

9 this approach, a highly unpredictable watermarking plane can be 

1£L* produced. Moreover, it can be reproduced exactly by knowing only 

W 

IB its four ^robust-watermarking-parameters' . These parameters are 

- &\ 

1Z| the initial state and the two coefficients of the congruence 

lpi algorithm, and the cryptographic key used by the encryption 

lfe; algorithm. These algorithms generally produce sequences of values 

f=T?< 

1§3 having eight-bits. Sixteen-bit values, referred to as a(i,j), 

lgj are generated by concatenating two of the sequential eight-bit 

lp values produced by the encryption algorithm. Each sixteen-bit 

l|l value so produced is linearly remapped to become an element of 

l|P the array defining the watermarking plane as follows: 

20 w(i,j) = 1 - 20[1 - a(i, j)/65535] (5) 

21 Additionally, all elements in the w(i,j) array, treated as an 

22 ensemble, are adjusted to have both a mean and median of l~p. 

23 Ease of reproduction of the resulting encrypted sequence is 

24 important for watermark .detection and demonstration techniques 

25 discussed subsequently. Other remapping or normalization 

26 techniques producing particular desired results are known to 

27 those familiar with the art. 
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2 The Property of Explicit Low Frequency Content 

3 Another important consideration is for an embodiment that 

4 exploits the property of explicit low frequency content. 

5 Significant high frequency content results when the watermarking 

6 plane is composed by placing a unique random value in every 

7 element. Although high frequency content is beneficial in making 

8 the watermark less visible, it also makes it vulnerable to attack 
gj for watermark damage or extinction. This is evident from the 

(0 following consideration. The highest pattern frequency achievable 

in a digitized image is obtained by replicating a pair of 

& adjacent pixels that have opposite extreme values. When the image 

3= is reduced in size, if image reduction filtering is used, the 

8 values of adjacent pixels are combined in a weighted average to 
|g form pixel values of the reduced image. If image decimation is 

jP used, pixels are selectively discarded. In either event, the high 

I* frequency content in the original image is lost in the reduced 

jP image. Any significant high frequency content in the applied 

9 watermarking plane becomes obliterated in the reduced image. 

20 Subsequent detection of the watermark imparted prior to the size 

21 reduction is very difficult if not impossible. The purposeful 

22 addition of low frequency content makes the watermark less 

23 vulnerable to this type of attack. However, the deliberate 

24 inclusion of significant low frequency content in the 

25 watermarking plane is another dichotomy. Its inclusion indeed 

26 makes the watermark less vulnerable to normal image manipulation 

27 and therefore more easily detectable. However, it generally makes 

28 the watermark more visible by producing a pattern with larger' 

29 features in the watermarking plane. It is generally preferable to 

30 add only a controlled amount of low frequency content. 
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1 The deliberate addition of low frequency content to the original 

2 watermarking plane is accomplished in one embodiment by employing 

3 the two-dimensional discrete Fourier transform. First, a 

4 reduced-size watermarking plane is formed whose elements are 

5 uniformly distributed random values in accordance with the secure 

6 sequence described above. For discussion purposes, a square plane 

7 w(\i,v) having 0<p<A-l rows and 0<v<A-l columns is used. The 

8 discrete Fourier transform of. the square plane is computed. Since 

9 all values of w(p,v) are real numbers, advantage can be taken 

\Qi from the complex-conjugate symmetry of its Fourier transform. The 

ljg complete Fourier transform can be specified as the array of 
W 

lj] complex numbers W(<j,t) having dimensions 0<a<A-l and 0<r<A/2, and 

lfU is symbolized as: 

Ifg W((T,T)=F[W(11,V)] (6) 

Ms 

1%] The frequency domain array W(<t,t) is remapped into an expanded 
array W(s,t), where: 

18 0<s<L-l, 0<t<L/2, and L=2 P A, 

19 thus enlarging the (a, r) -space by the factor 2 P in each dimension 

20 forming the larger (s, t) -space . If W(a,T) is defined such that 

21 VI (0,0) is the coefficient of the constant or "zero-frequency" 

22 term, then: 

23 W(L-s, t)=W(h-a,t) , (7) 

24 and 
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1 W(S,t)=W(<T,T) , (8) 

2 for 

3 0<s=cr<A/2 and 0<t=T<A/2, (9) 

4 and 

5 W(s,t)=0 (10) 

6 for all other values of s and t. This technique is herein 

7 referred to as "zero-padding." 



£3 



The inverse Fourier transform of W(s,t) provides the modified 
watermarking plane w(m,n) having 0<5]7<L-1 rows and 0<n<L-l 



columns. If, for example, p = 2 and A=512, then w(m,n) is a 

it* square array having 2048 rows and columns. More importantly, 

ry 

12U however, w(m,n) has an assured low frequency content with a 

13^ minimum period (2 P =2 2 =4) four times longer than the minimum 

IG period possible in a 2048 2 image plane. Since its generating 

if*, kernel, w(ji,v), contains 262,144 random values taken from a 

1&? secure sequence, its vulnerability to attack by brute force 

lfi replication is relatively small. In a case where a thus marked 

18 image appears to be vulnerable, its kernel can easily be made 

19 larger. Still lower frequency content can be impressed by using a 

20 p=3, making the highest frequency to be one eighth of the 

21 original highest frequency. The preferred embodiment uses p=2 so 

22 as not to over employ low frequency content that may cause the 

23 watermark to become undesirably visible. 



24 The values of some elements of the generated watermarking plane 

25 so far produced may exceed one. Since each value is to be used as 

26 a multiplier of pixel brightness, it is therefore possible to 
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1 produce a multiplied pixel brightness that is greater than one 

2 [i.e. greater than a maximum brightness], which is a brightness 

3 that can not physically be displayed. In this event, any 

4 multiplied pixel brightness greater than one would have to be 

5 clipped to the maximum that can be displayed. The preferred 

6 embodiment, however, employs an additional process step to avoid 

7 the possible need for clipping. Before the generated modified 

8 watermarking plane is used, its elements, forming an ensemble, 

9 are adjusted to make both their mean and median values equal to 
10 1-p and the maximum value equal to 1. With these adjustments, the 
IJjj requirement that, 

12* l>w(i,j)>(l-2$), (11) 

fU 
\* 

l|g f°r all i and j is satisfied. 
0 

1|Q At this point, it is sometimes advantageous to "hard clip" the 

1ST elements. In this situation, elements with values greater than or 

113 equal to 1~P are set to 1, and elements with values less than 1-|3 

17 are set to 1-20. Hard clipping normally increases the probability 

18 of detecting a watermark, but unfortunately, it also tends to 

19 make watermarking artifacts more visible in the marked image. 
20 

21 The Property of Plane Expansion by Tiling 



22 The fact that the .watermarking plane w(m,n) is produced as the 

23 result of an inverse' discrete Fourier transform is very useful. 

24 If the watermarking plane is not large enough to cover the entire 

25 unmarked image, if L<I or L<J, it can be enlarged seamlessly by 

26 tiling replication downward or to the right to make a plane as 
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large as desired, with each tiled extension adding an additional 
4,194,304 elements. For the example dimensions used here, tiling 
replication is: 



w (m' , n' ) = w (m, n) , 



(12) 



where, 



m' = (2048p) + m, 



(13) 



n' = (2048q) + n, 



(14) 



and p and q are non negative integers. 

In one embodiment, a watermarking plane is formed following the 
steps 202-216 shown in Figure 2. These steps are herein referred 
to as the *ideal interpolator watermarking plane generating 
method' . Firstly, an eight-bit pseudo-random sequence is 
generated, 202. The resulting sequence is encrypted to form a 
secure sequence of eight-bits values, 204. Sixteen bit integer 
samples are formed by concatenating two abutted values from the 
secure sequence, 206. The sixteen bit integer samples are 
linearly remapped and formed into a w(}i,v) array such that, 



208. The discrete Fourier transform frequency domain array W(<t,t) 
is computed from w(jj,v), 210. The W(g,t) coordinates are expanded 
by zero-padding to form expanded frequency domain array W(s,t), 
212. The preliminary watermarking plane array w(m,n) is computed 
by taking the inverse discrete Fourier transform of W(s,t), 214. 
The elements of the preliminary array w(m,n) are adjusted to 



lZw(v, W>(l-2p) , 



(15) 
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collectively have a mean and median of (1-p) and a maximum of 1, 
216a. Alternatively, the elements w(m,n) are hard clipped to have 
only values of 1 or 1-2(3, with a median of 1-p, 216b. The 
resulting adjusted array w(m,n) is the watermarking plane with 
elements that are brightness multiplying factors to be used for 
adjusting corresponding pixels of the image being watermarked. 

The method presented here, employing forward and inverse discrete 
Fourier transforms to generate the watermarking plane, is an 
"ideal interpolator" with assured low frequency content. Other 
methods known to those skilled in-the-art are available. These 
include methods that use two-dimensional interpolation filters 
that can similarly be employed to produce acceptable results. 

The generated watermarking plane is then imparted onto the 
original unmarked digitized image. Figure 3 shows an embodiment 
for the steps of watermark imparting. First, the watermarking 
plane is expanded by tiling to completely cover the image being 
watermarked, 302. This forms a one-to-one correspondence of an 
element in the expanded watermarking plane and a pixel in the 
original image. The brightness values of each pixel in the 
original image are multiplied by the value in its corresponding 
element in the expanded watermarking plane, 304. The resulting 
image with the new brightness values forms the watermarked image. 
The relative visibility of the watermark in the image is observed 
in relationship to the desired visibility classification level 
marking. If the marking is more visible than specified the steps 
of Figures 2 and 3 are repeated for a lower modulation strength. 
A watermark created with a lower modulation strength is generally 
less easily detected and demonstrated to exist. One the other 
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1 hand, if the resulting watermark is less visible than specified, 

2 the steps of Figures 2 and 3 may be repeated to provide a 

3 watermark with a higher modulation strength. A watermark created 

4 at a higher modulation strength is generally easier to detect and 

5 have its existence demonstrated. Once imparted, an invisible 

6 watermark only serves its purpose if it can be detected and shown 

7 to exist. 

8 FINDING AN INVISIBLE WATERMARK HIDDEN IN A MARKED IMAGE 

|| It is most desirable to demonstrate the existence of the 

ICS watermark with a visible image having bold features. This is 

herein employed using an image array called a " visualizer ." 

lfU Demonstration of the existence of the watermark imparted in 

l|g accordance with the^ present invention requires a regeneration of 

14 the watermarking plane with which it was marked. This can 

Q 

l|g generally only be performed by the marker and/or marking entity 

li>- who alone knows the four parameters making up this application's 

SI 

I**) " robust-watermar king-parameters" ' . Knowledge of these parameters 
is required for generating the robust random sequence used in 

19 forming the watermarking plane. From these four parameters the 

20 robust random sequence is reformed. Values of the sequence are ' 

21 used to define the values of the elements. If a linear remapping 

22 process was employed in the generation of the watermarking plane, 

23 the element values are linearly remapped using that same process 

24 to redefine the expanded watermarking plane. The thus reformed 

25 expanded watermarking plane is used in conjunction with the 

26 visualizer to demonstrate the existence of the expanded 

27 watermarking plane in the image. This is accomplished as 

28 described subsequent to an overview of watermark detection 

29 considerations . 
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1 Finding an invisible watermark hidden in a marked digitized image 

2 is a relatively difficult problem, and it is made more so by 

3 . manipulations of the marked image that may have occurred. The 

4 watermark survives and is detectable for image manipulations that 

5 in themselves do not damage the image beyond usability. The 

6 detection method of the present invention can find an imparted 

7 watermark with a high degree of certainty in nearly all such 

8 cases. A significant advantage of the present method is that 

9 watermark detection does not require access to a copy of the 

18i entire original image. In most cases, all that is required is the 

lfl watermarking plane used for imparting the watermark on the image, 

l^j A perfect copy of the watermarking plane is reconstituted from 

1^ its four defining parameters. If a copy or if even only a 

1{4 fragment of the original image is available, detection can have a 

15? somewhat higher probability of success. 

P 

lf§| Reorienting and Resizing the Watermarking Plane 

10 A first consideration in finding a watermark is to determine how 

18 and by how much the marked image may have been manipulated. It 

19 may have been reduced in size. A size reduction may even have 

20 been performed nonlinearly, such that its horizontal and vertical 

21 dimensions may have been reduced by unequal factors. The image 

22 may also have been rotated, not by an obvious ninety degrees, but 

23 by a small angle. Facilitating this determination is the 

24 knowledge that pixel values in the unmanipulated marked image are 

25 directly related to corresponding elements in the watermarking 

26 plane. If a significant fragment of the original image is 

27 available, a fragment of the unmanipulated marked image can be 

28 reconstructed. Either the reconstituted watermarking plane or a 
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1 reconstructed fragment of the marked image is a suitable 

2 "correlation reference plane." 

3 An overview of the steps of reconstructing a manipulated 

4 watermarked image is shown in Figure 4. First, the watermarking 

5 plane used for imparting the watermark onto the image is 

6 regenerated from the four ^robust-watermarking-parameters' 

7 generally only known to the marker and/or the marking entity, 

8 402. Secondly, the marked image is resized and rotated to its 

9 known original dimensions, 404. Thirdly, the resized and rotated 
lgg image is aligned with the expanded regenerated watermarking plane 
ll| such as to provide one-to-one correspondence of the elements of 

each with the elements of the other, 406. 

rv 

l$k In an actual implementation the steps of reorienting and resizing 

14* the marked image may be broken into a coarse placement followed 

lfy by a fine alignment. The coarse placement is performed by visual 

lof inspection of a displayed copy of a portion or the complete 

1 Wl marked image overlaying a corresponding portion or complete 

18^ correlation reference plane. The correlation reference plane is 

19 reoriented and resized to the size and orientation of the marked 

20 image by axis reduction or expansion,, translation and/or 

21 rotation. This is accomplished using techniques well known to 

22 those skilled in the art. The coarse placement generally brings 

23 the correlation reference plane to within 4 percent of the 

24 manipulated. marked image's size and within four degrees of its 

25 orientation. 

26 Figure 5 shows the steps for an embodiment for performing coarse 

27 placement. Both the marked image and the correlation reference 

28 plane are displayed on a common display, 502. The vertical axis 
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and horizontal axis magnification, offset and angular rotation of 
the correlation reference plane display are varied to make the 
displayed correlation reference plane closely overlay the 
corresponding portions of the displayed manipulated marked image, 
504. The values of the magnification/reduction factors, 
horizontal and vertical offsets and angle of rotation are noted 
and stored, 506. The entire marked image is rescaled, translated 
and rotated by the inverses of the noted values so that it 
visually matches the correlation reference plane, 508. The so 
coarsely manipulated reconstituted marked image is further 
manipulated to perform the fine alignment. 

According to the Fourier Shift Theorem, Rotation Theorem and 
Scaling Theorem, the properties of translation, rotation and 
scaling transcend the Fourier transformation of an image, and, if 
present in w(m,n) f each will also be present (or, in the case of 
scaling, its reciprocal will be present) in W(s,t). This is 
useful to determine a more precise angle of rotation, horizontal 
and vertical scale factors, and translation offsets of the 
correlation reference plane relative to the marked image. This is 
accomplished by first constructing a three-dimensional "array of 
phase-correlation maxima." The. three axes of the array correspond 
to the horizontal scale factor, the vertical scale factor, and 
the angle of rotation of the correlation reference plane relative 
to the marked image. Phase-correlation is defined as follows. Let 
W(s,t) be the discrete Fourier transform of the correlation 
reference plane, U(s,t) be the discrete Fourier transform of the 
marked image u(m,n) f and U*(s,t) be the complex conjugate of 
U(s,t). The phase-correlation plane p(m,n) is computed using the 
relationship : 



DOCKET NUMBER: YOR919960153US4 



-32- 



# 



1 _, ^(5,0^/^(5,0 

^*) =i? W,ot/*(*,oi ] - (16) 

2 The value at each array point is the maximum magnitude of the 

3 corresponding phase-correlation plane. It is computed using an 

4 incrementally rescaled and rotated correlation reference plane. 

5 Any one of the color planes. of the marked image usually suffices 

6 as the required array u(m,n). Interpolating among the values of 

7 the three-dimensional array yields coordinates of the 

8 greatest-of-the-greatest phase-correlation rmaxima. From these 

£D coordinates, values of the horizontal and vertical scale factors 

lj^j and angle of rotation of the correlation reference plane relative 

ltti to. the marked image are directly read. The correlation reference 

12y plane is. then rescaled and rotated to more precisely align it 

lW* with the manipulated marked image. A final phase-correlation 

lp evaluation is made to determine the relative ' horizontal and 

113 vertical offsets of the modified correlation reference plane 
lSJ . relative to the manipulated marked image. Finally, the entire 

1$} marked image is rescaled, translated and rotated in accordance 

H 

lg^ with the inverses of the measured values to restore it to its 

19 original size and orientation. The thus modified marked image is 

20 now ready for use in the detection and demonstration process to 

21 . show the. existence of the watermark in the manipulated marked 

22 image . 

23 In one embodiment the fine alignment of the correlation reference 

24 plane relative to the marked image is performed by evaluating a 

25 . ...three-dimensional array of phase-correlation maxima, and then 

26 interpolating within that array to find the location of the 

27 maximum of those maxima. The axes of the array are the horizontal 

28 magnification, vertical magnification and angular rotation that 
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1 are systematically applied to the correlation reference plane. 

2 All combinations of the following incremental steps define the 

3 values of the coordinates of the array. The vertical axis of 

4 w(m, n) is magnified/reduced from 96% to 104% of its original size 

5 in 2% increments. In similar fashion the horizontal axis of 

6 w(m, n) is magnified/reduced from 96% to 104% of its original size 

7 in 2% increments. Also in similar fashion w(m,n) is rotated 

8 relative to its original orientation from -5 degrees to +5 

9 degrees in 2 degree steps. At each combination of vertical 

10 magnification, horizontal magnif ication, and angular rotation of 

JL the correlation reference plane, the phase-correlation plane 
Q 

J p(m,n) is recomputed as above. The maximum of the point values 

] p(m*,n*) in the plane is stored into the three-dimensional array 

Lfte of phase-correlation maxima at coordinates corresponding to each 

jjjjl of the incrementally adjusted values of vertical magnification, 

l£3 horizontal magnification, and angular rotation. 

3(3 A flow diagram of this embodiment is shown in Figure 6. Those 

u 

skilled in the art know there are many satisfactory algorithms 

3 available to magnify/reduce and rotate digitized images. Any one 
U 

20 of those algorithms can be used for manipulation of the 

21 correlation reference plane in the following description. As 

22 described above, the discrete Fourier transform of the marked 

23 image U(s,t) is formed, 602. Initial values are set for stepping 

24 variables vertical magnification, Vm=0.96, horizontal 

25 magnification, Hm=0.96, and angular rotation, Ar=-5°, 604. The 

26 correlation reference plane is vertically magnified/reduced 

27 according to Vm, 606. The so adjusted plane is then horizontally 

28 magnified/reduced according to Hm, 608. The so adjusted plane is 

29 then rotated according to Ar, 609. The discrete Fourier transform 

30 of the so adjusted plane W(s,t) is formed, 610. The 
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phase-correlation plane p(m,n) is calculated using the 
relationship of equation (16), 611. The p(m,n) plane is examined 
to find the coordinates (m*,n*) of its maximum value, 612. The 
coordinates (m*,n*) and p (m*,n*) are stored in the 
three-dimensional array being formed. The three-dimensional array 
is indexed by Vm, Hm and Ar, 613. The value of Ar is examined, 
614. If it is less than plus five degrees, it is incremented by 
plus two degrees, 615, and steps 609-614 are repeated until Ar is 
found to be plus five degrees in step 614. When Ar is found to be 
plus five degrees in step 614, the value of Hm is examined, 616. 
If Hm is less than 1.04, it is incremented by 0.02 and Ar is 
reinitialized to minus five degrees, 617. Steps 608 to 616 are 
repeated until Hm is found to be 1.04 in step 616. When Hm is 
found to be 1.04, Vm is examined, 618. If Vm is found to be less 
than 1.04, it is incremented by 0.02, and Ar is initialized to 
minus five degrees, and Hm is initialized to 0.96, 619. Steps 606 
to 618 are repeated until Vm is found to have a value of 1.04 in 
step 618. When Vm is found to be equal to 1.04, the values of the 
three-dimensional array are interpolated to find the maximum of 
the maxima peaks, 620. The resulting coordinates of the maximum 
of maxima peaks provide the final values for the vertical 
multiplier, the horizontal multiplier and the rotational angle 
for best alignment of the manipulated marked image with the 
correlation reference plane. The corresponding resulting values 
of m* and n* of the maximum of maxima provide the offset 
displacements of the manipulated marked image relative to the 
correlation reference plane. The manipulated marked image is then 
rescaled by the inverses (reciprocals) of the found vertical and 
horizontal multipliers. It is rotated by the inverse (negative) 
of the found angular rotation, and is offset by the inverses 
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(negatives) of m* and n*, 622. This completes the fine setting 
process of reorienting and resizing. 

It will be apparent to those skilled in the art that either the 
correlation reference plane or the manipulated marked image can 
be resized and reoriented to bring one into alignment with the 
other. The preferred embodiment resizes and reorients the 
manipulated marked image to bring it into alignment with the 
correlation reference plane, and hence into element-to-element 
alignment with the watermarking plane. 

Detecting the Watermark in a Marked Image 

The process of watermark detection is designed to produce a 
visibly recognizable small image as its end product. The 
recognizable end product is obtained in a procedure which depends 
upon the existence and knowledge of the watermark based on the 
robust random sequence. The process exploits the extremely 
sophisticated and not yet completely understood pattern 
recognition capabilities of the human visual system. It is 
through this exploitation that defeating the imparted watermark 
becomes much more difficult. A small rectangular array, called a 
selector, is conceived to implement the detection process. The 
selector array size must be much smaller than the pixel array of 
the marked image to which it is being applied. This is to allow 
overlaying the selector on the image hundreds of times without 
overlapping. The selector array should be made large enough that 
a pixel array having the same dimensions could contain a 
recognizable binary image. More complex embodiments use a color 
rather than binary image as a reference. A selector having 32 
rows and 128 columns is used in an embodiment described herein. 
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1 It is applied to a marked image that has more than one million 

2 pixels. 

3 The selector is used to locate rectangular clusters of pixels in 

4 the marked image and corresponding clusters of elements in the 

5 reconstituted watermarking plane. The clusters are randomly 

6 scattered non-overlapping positions. Random scattering of the 

7 clusters is done to further frustrate attempts to defeat 

8 watermark protection. Each element of the selector contains one 

9 or more devices associated with variables that serve to store 

ltjp partial results of the watermark detection scheme. One embodiment 

l|| uses two selector devices, one called a "coincidence counter" and 
the other a "non-coincidence counter." All coincidence counters 

.y and non-coincidence counters are set to a zero value before the 

14* detection process is begun. 

Q 

l£3 A variable, called a statistically related variable, is defined 

iff which statistically relates an attribute of an element being 

IW considered to the attributes of its neighboring elements. For 

l|5 each pixel in the marked image a first variable is computed for 

19 that pixel and a second variable is computed for that pixel's 

20 corresponding element in the reconstituted watermarking plane. A 

21 positive test results when the computed first variable has the 

22 same result, or a nearly or statistically deemed equivalent 

23 result, as the computed second variable. If the results are 

24 deemed to be different, the test result is deemed to be negative. 

25 The first variable is recomputed and compared with the second 

26 variable for each of that pixel's color planes. The coincidence 
.27 counter associated with that selector element is incremented by 

28 unity for each color plane producing a positive result and the 

29 non-coincidence counter is incremented by unity for each color 
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1 plane that produces a negative result. The purpose of each 

2 element's coincidence and non-coincidence counters is to 

3 associate with that element a confidence level of the watermark' s 

4 identification with the random sequence known only to the marker 

5 and/or the marking entity. The quantified confidence level for 

6 each element is derived from the values in that element's 

7 coincidence and non-coincidence counters, and is called a 

8 coincidence value. 



9 For a tristimulus color image and for each cluster of pixels, the 

1@ range of each coincidence counter value is from zero to plus 

l|p three. A zero is obtained if the test results were negative for 

all three color planes. A plus three is obtained if the test 

lpj results were positive for all three planes. The range of each 

lit non-coincidence counter is also from zero to plus three, but 

IS conversely, a zero is obtained if the test results for all three 

l^g planes were positive and a plus three is obtained if the test 

IH results of all three planes were negative. The count in each 

l|sj coincidence counter is the accumulated sum of the counts of 

l£* positive results for corresponding pixels at each cluster 

20 location, and the count in each non-coincidence counter is the 

21 accumulated sum of the counts of negative results for 

22 corresponding pixels at each cluster location. A coincidence 

23 counter value larger than the value of its corresponding 

24 non-coincidence counter is associated with a partial watermark 

25 detection. A composite of coincidence counter values greater than 

26 their corresponding non-coincidence counter values for a 

27 preponderance of the selector's elements results from and 

28 corresponds with a detected watermark having a high confidence 

29 value. 
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In an embodiment the test results and/or the comparison are 
performed by subtraction operations. In a particular embodiment 
the attribute used is the pixel's brightness values. The 
statistical relationship is in regard to the average brightness 
value of the neighboring pixels. In this case, watermark 
detection proceeds with the steps shown in Figure 7. A selector 
array size is selected, 702. In this example, the selector array 
size is 32 by 128 elements. All the coincidence and 
non-coincidence counters are initialized by setting them to read 
zero, 704. A specified particular element of the selector is 
placed on an initial position of the expanded watermarking plane, 
706. The particular first element is often the selector element 
that is at its upper leftmost corner. This particular element 
also locates a corresponding pixel and its components in all the 
color planes of the marked image when the marked image is aligned 
with the expanded watermarking plane. 

The following portion of the detection schema is repeated 
iteratively for all selector elements, for all color planes of 
each pixel, and for all selected clusters. The next two eight-bit 
integers are chosen from the regenerated robust random sequence, 
708. When the schema is started for the first selector element, 
the next two eight-bit integers chosen in this step 708 are 
actually the first two eight-bit integers of the robust random 
sequence. The two eight-bit integers are scaled to form random 
horizontal and vertical offsets from the initial or previous 
selector location, and the selector is moved to that position, 
710. The selector element sequence is reset to the coordinates of 
the initial particular selector element, 711. This selector 
element is used to locate the corresponding particular element in 
the watermarking plane, 712. The average magnitude of its 
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1 neighboring elements in the watermarking plane is computed, 713. 

2 In the example, this is the average of the magnitudes of the 

3 particular element's neighbors that lie in an 11 by 11 square of 

4 elements with the particular element at the center of the square. 

5 If the selector element is too near an edge of the watermarking 

6 plane to be at the center of its neighborhood, the square 

7 neighborhood is moved to encompass the particular element's 

8 nearest 120 existing neighbors. 

9 The next color plane is chosen, 714. In the beginning of this 
iterative schema this next color plane is actually the first 

1*0 color plane. In the case of a monochrome image this is the only 

1|4 color plane. The coordinates of the particular selector element 

l9J are used to locate a corresponding pixel color component in this 

lj4j next color plane, 715. The average brightness of the neighboring 

16^ 120 pixel color components is computed, 716, in a manner 

lgj identical to that stated above for watermarking plane elements. 

1^ The values of the particular watermarking plane element and the 

1§j corresponding pixel color component are compared to their 

1^ respective neighborhood averages. If both values are equal to or 

20 greater than their respective neighborhood averages, 717, or if 

21 both values are less than their respective neighborhood averages, 

22 718, the coincidence counter of that particular selector element 

23 is incremented, 719a. If one value is less than its respective 

24 neighborhood average and the other value is equal to or greater 

25 than its respective neighborhood average, the non-coincidence 

26 counter of that particular selector element is incremented, 719b. 

27 The magnitude of the value in each coincidence counter relative 

28 to the magnitude of the value in its corresponding 

29 non-coincidence counter is associated with the probability of 

30 watermark sequence validation. 
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A determination is made if all color planes were chosen for 
testing their corresponding brightness value with regard to its 
neighboring average, 720. If not, the process returns to step 714 
for choosing the next color plane. Steps 715 to 720 are repeated 
for this color plane. This is continued until step 720 indicates 
the all color planes are tested. When the last (or only) 
color-plane is tested, a determination is made if every element 
for that selector was chosen, 724. If not, the next selector 
element is chosen, 726. Generally, the next element is the next 
right-wise adjacent element on that row. If there is no next 
adjacent element on that row, the next element is the left-most 
element in the next selector row. This next selector element 
becomes the new particular element. Steps 712-724 are repeated 
until all selector elements are chosen and tested. When it is 
determined in step 724 that all elements have been chosen, a 
determination is made if all non-overlapping selector locations 
have been chosen, 728. If not, steps 708 through 728 are repeated 
for all selector elements and marked image color planes. When it 
is determined in step 728 that all selector locations are tested, 
all coincidence counters have their test result values. 

Figure 8 shows a random multiple totality of positions of the 
selector 810 in a selector plane 802 resulting from an 
implementation of the process of Figure 7. Figure 8 shows a 
watermarking plane 804 and three color planes 806-808 of the 
marked image. The first selector element acted upon is often the 
top leftmost element 812 of the selector in each of the selector 
positions. It is noted that although each selector position is 
randomly offset from previously chosen positions, the positions 
do not overlap each other. 
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1 The values contained within each coincidence and non-coincidence 

2 counter associates with their corresponding selector element a 

3 confidence level of the watermark's identification with the 

4 random sequence known only to the marker and/or the marking 

5 entity. The watermark is considered to be detected if a 

6 preponderance of the differences of coincidence counter values 

7 less their respective non-coincidence counter values are non 

8 negative. Thus, an examination of the .totality of these non 
negative differences explicitly suffices for declaring the 

lffl watermark detected or not detected. Indeed, this can be 

if) 

ljj considered as the end of the watermark detection technique. 

lSjj Those skilled in the art will recognize that it is possible to 

1§3 mathematically derive a "probability of watermark detection/' in 

1^ which the "probability of watermark detection" is greater than 

lp zero and less than one (where a value zero represents certainty 

l^i of the absence of a watermark and a value one represents 

lf4 certainty of its presence) , based only on the coincidence and 

18 non-coincidence counter values and assuming only the property of 

19 uniform distribution of the random brightness multiplying 

20 factors. However, alternative embodiments recognize that a 

21 "preponderance" of differences being non negative is an inexact 

22 measure, at best. Clearly, if only a simple majority of the 

23 differences are non negative, whether the watermark is detected 

24 or not is at best a judgment call. Most likely it would be 

25 conceded as not having been a detection. To assist in this 

26 judgment, the present invention exploits the ability of the human 

27 visual system to recognize a pattern in a cluttered field. This 

28 is accomplished by forming a binary image called a visualizer. 

29 The visualizer is formed to have the same dimensions as the 
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1 selector (e.g., 32x128 pixels). A clearly recognizable pattern is 

2 placed into the visualizer. A typical visualizer pattern is shown 

3 in Figure 9, 900. The black border surrounding the visualizer is 

4 not considered to be part of the visualizer pattern. The pattern 

5 is an arrangement of blocks of black and white pixels forming an 

6 easily recognizable pattern. A typical pixel, 902, is at the 

7 lower ending of the image of a C. The visualizer image is 

8 entirely white except for pixels making the letters IBM, 904, the 

9 copyright logo, 906, and the visualizer frame, 908. 

The visualizer pattern is used to provide a visual image of the 
actual degree of "preponderance" of coincidence counters being 
non negative. The method steps diagrammed in Figure 10 are used 
to provide a watermark signature in relation to the visualizer 
pattern. The watermark signature is derived by using the 
visualizer pattern in combination with the coincidence counter 
difference data to form what is herein referred to as the 
I tO ^visualizer-coincidence image' . 

CP 

S3 In one embodiment, the visualizer-coincidence image is formed 

y with the steps shown in Figure 10. A visualizer pattern is formed 

20 having a pixel array equal in size to the element array of the 

21 selector, 1002. The visualizer array consists of white and black 

22 pixels, where white is given the value one and black the value 

23 zero. All elements of the selector array will be examined to 

24 determine the pixel content of the visualizer-coincidence image. 

25 To do this, the selector element sequence is reset and the first 

26 element of the sequence is chosen, 1004. For the chosen selector 

27 element, the count in its corresponding non-coincidence counter 

28 is subtracted from the count in its corresponding coincidence 

29 counter, forming a difference, 1006. The sign of the difference 
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1 is tested, 1008, and if it is negative the corresponding pixel of 

2 the visualizer is inverted (white is changed to black, and black 

3 to white) and placed into the corresponding pixel of the 

4 visualizer-coincidence image, 1010b. If the sign is positive, the 

5 corresponding pixel of the visualizer is placed unmodified into 

6 the corresponding pixel of the visualizer-coincidence image, 

7 1010a. The selector element sequence is tested to see if all 

8 elements have been chosen, 1012, and if not , the next element is 

9 chosen, 1014, and steps 1006 to 1012 are repeated. If all 

lg-j selector elements have been chosen, the visualizer-coincidence 

ll3 image is displayed, 1016. A judgment is made as to whether the 

lgj pattern in the visualizer-coincidence image is recognized as a 

l!r reproduction of the visualizer pattern, 1018. If it is 

l|t recognized, the watermark is positively detected, 1020a. If not, 

1§3 the watermark is not detected, 1020b. 

□ 

16* It is evident to those skilled in the art that if only the sign 

111 of the difference between the count in a coincidence counter less 

lP the count in its corresponding non-coincidence is to be used in 

19 constructing the visualizer-coincidence image, then only one 

20 counter would have been needed for each selector element. In that 

21 case, step 719a of Figure 7 would read "Increment the counter of 

22 Selector's Element," and step 719b would read "Decrement the 

23 counter of Selector's Element." 

24 Figure 11 shows a detection, 1102, resulting from the visualizer 

25 of Figure 8 for an imparted watermark made at a modulation 

26 strength of 1%. As previously stated in all cases the black 

27 border is not part of the visualizer-coincidence image. A 

28 stronger replication of the visualizer, 1202, resulting for an 

29 imparted watermark made at a modulation strength of 2% is shown 
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1 in Figure 12. A still stronger replication of the visualizer, 

2 1302 , resulting for an imparted watermark made at a modulation 

3 strength of 4% is shown in Figure 13. 

4 An attempt to detect a watermark in an image that does not have 

5 one, or in an image for which the watermarking plane cannot be 

6 reconstituted, produces a visualizer pattern that is an 

7 unrecognizable random melee.. Figure 14 shows a typical 

8 visualizer-coincidence image, .1402, when a watermark is not 

9 detected. This results when many visualizer pixels are subjected 
to inversion. A preponderance of pixels not requiring inversion 
indicates watermark detection. This method in fact has an 

lgj extremely low probability of false-positive detection. Even in a 

ljf highly textured marked image, the visualizer pattern should be 

ljfe clearly recognizable to signify a watermark detection of very 

l?r high credibility. 

?. 

9 

Clearly, more information is present in the coincidence and 

l|l non-coincidence counter values than has been exploited above, 

l& where only the algebraic sign of their difference has been used. 

19 An alternative method of converting the visualizer image into a 

20 visualizer-coincidence image uses the magnitude of each 

21 coincidence counter value and that of its corresponding 

22 non-coincidence counter. If C(i',j') is the value of the 

23 coincidence counter associated with selector element i' ,j' and 

24 C y (i / ,j' ) is the value of the corresponding non-coincidence 

25 counter, then the normalized magnitude of their difference 

26 e(i' ,j f ) is: 

27 e(i',j')= C(i*j*)/[C(i\j*)+C'(i' (17) 
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1 when C(i \j *)+C y (i',j')>O r (18) 

2 and: 

3 e(i f ,y)= 1/2, (19) 

4 when C (i \ j ') +C 1 (i ' , j ') =0 . (20) 

5 In this case, the visualizer image is converted into a 

6 visualizer-coincidence image by replacing each pixel in the 

7 * visualizer image with the corresponding value of e(i',j')f when 

8 the visualizer pixel value is one; and by l-e(i',j')/ when the 

9* visualizer pixel value is zero. Notice that the 

ICS visualizer-coincidence image is no longer a binary image, but 
■ 

ltj includes gray shades ranging from black to white. The judgment as 

lzfj to whether the pattern placed in the visualizer is recognizable 

lE in the visualizer-coincidence image is the same as before, and an 

143 attempt to detect the presence of a known watermark in an image 

lfg not having one, or in an image having one but for which the 

16$ watermarking plane cannot be precisely reconstituted, also still 

l|p produces an unrecognizable random melee in the 

li3 visualizer-coincidence image. 

19 Thus, this scheme makes more use of the actual values in the 

20 coincidence and non-coincidence counters. It still employs a 

21 black and white element visualizer image pattern wherein each 

22 element is either black or white (zero or one) . However, the 

23 resulting elements of the visualizer-coincidence image have 

24 values ranging between zero and one such that when displayed it 

25 has various levels of shades of gray. The gray level depends on 

26 the counter data. 
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1 An embodiment of this alternative scheme is shown in Figure 15. A 

2 visualizer pattern is formed having a pixel array equal in size 

3 to the element array of the selector, 1502. The visualizer array 

4 consists of white and black pixels, where white is given the 

5 value one and black the value zero. All elements of the selector 

6 array will be examined to determine the pixel content of the 

7 visualizer-coincidence image. To do this, the selector element 

8 sequence is reset and the first element of the sequence is 

9 chosen, 1504. For the chosen selector element, the ratio of the 
0 count in its corresponding coincidence counter to the sum of the 

counts in its corresponding coincidence and non-coincidence 

p counters is computed, 1506. The color of the corresponding 

jf| visualizer pixel is tested, 1508, and if it is black, the ratio 

jjjj subtracted from one is placed into the corresponding pixel of the 

jjjg visualizer-coincidence image, 1510a. If the visualizer pixel is 

^ white, the ratio is placed unmodified into the corresponding 

gjj pixel of the visualizer-coincidence image, 1510b. The selector 

If element sequence is tested to see if all elements have been 

on 

19 chosen, 1512, and if not, the next element is chosen, 1514, and 

2tf steps 1506 to 1512 are repeated. If all selector elements have 

21 been chosen, the visualizer-coincidence image is displayed as a 

22 high contrast monochrome image, 1516. A judgment is made as to 

23 whether the pattern in the visualizer-coincidence image is 

24 recognized as a reproduction of the visualizer pattern, 1518. If 

25 it is recognized, the watermark is positively detected, 1520a. If 

26 not, the watermark is not detected, 1520b. 



27 THE IMPLEMENTATION OF BRIGHTNESS MODIFICATION BY ADDITION INSTEAD 



28 OF MULTIPLICATION 
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1 An alternative, and equivalent, form for modifying pixel 

2 brightness is to change the brightness by adding to or 

3 subtracting from the component Y(i,j) a different small random 

4 value e(i,j) . As before stated, 1 < i < I and 1 < j < J are the 

5 row and column indices of the pixel location in the image. To 

6 help make the brightness variation less visible, is made 

7 proportional to the original brightness of the component, thereby 

8 making the change smaller in darker areas of the image where the 

9 human eye is more discerning of changes in brightness. Thus, 

10 = 5 (i, j) Y (i, j) , where 5(i,j) is a value selected from an 

ljp array of random values that may have the range -0.5 < 5(1, j) < 

l&j 0.5. The modified component Y r = Y (i, j) + e (i, j) = 

15^ Y (i, j) +5 (i, j) Y (i, j) . To alter only the brightness of each pixel 

rsi 3 

3 a i 

ljjf in a color image, the ratios of its components must be preserved. 

153 If the color components of the unaltered pixel are X(i,j), 

s 

l|g Y(i,j), and Z(i,j), and the color components of the brightness 

If altered pixel are X'(i,j), Y'{i,j), and Z'(i,j), then 

m X' /X(i,j) = Z' (i,j)/Z(i,j) = Y' /Y(i,j) =l+5(i,j). It 

if 3 is evident that this is equivalent to multiplying the brightness 

20 of the each color component by 1+6 since X y {i,j)= 

21 X(i,j) [l+6(i,j) ] , Z'(iJ) = Z(i,j) [l+6(i,j) ] , and Y'(iJ) = 

22 Y (i,j) [1+5 ] . If the random values 1+6 (i,j) are set equal to 

23 w(i,j) as defined before, the two methods are identical. In 

24 summary, the modification of pixel brightness by an additive 

25 value that is proportional to pixel brightness while preserving 

26 the ratios of the color components of the pixel is equivalent to 

27 modifying the brightness of the pixel by multiplication. An 

28 additive and multiplicative modulation can have a different 
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1 effect only if the ratios of the color components of the pixel 

2 are allowed to change. 

3 USING A BLURRING FILTER BEFORE ATTEMPTING WATERMARK DETECTION TO 

4 IMPROVE THE PROBABILITY OF DETECTION 

5 Watermark detection may be enhanced in accordance with the 

6 present invention as described hereinafter in a manner that is 

7 adaptable for use of any of many watermarking techniques. It is 

8 most particularly adaptable to a watermarking technique employing 
U 1 a watermarking plane. Thus, although the enhancement of the 

1G>3 detection technique is adaptable to many watermarking techniques, 

II; it is most easily described and adaptable to the watermark 

lflj imparting and detecting methods described previously herein. 

li As described above for particular embodiments, watermarks are 

imparted into an image by multiplying the components of each 

life pixel of the image by the linearly remapped values of the 

m 

ifj watermarking plane, w(i,j) 9 where l>w (i , j )>(l-2$) , i is the 
U 

17 value's row index, j is the value's column index, and P is the 

18 modulation strength of the watermark. Additionally, all elements 

19 in the generated watermarking plane, treated as an ensemble, are 

20 adjusted to have a mean and median of 1-p- In other watermark 

21 embodiments this is accomplished by addition and/or subtraction 

22 operations. 

23 A method for improving the detection of the imparted watermark in 

24 a marked image and, more specifically, in a derived copy of a 

25 marked image employs use of a two-dimensional blurring filter 

26 prior to an attempted detection. A blurring filter is also called 



DOCKET NUMBER: YOR919960153US4 



-49- 




1 a low-pass filter in signal-processing terminology. Application 

2 of the blurring filter is advantageous in that it reduces 

3 high-frequency noise content among the color components in the 

4 marked image while leaving low-frequency content relatively 

5 unaltered. 

In the example embodiment, since the watermark, as imparted into 
the image, has the appearance and behavior of a two-dimensional 
noise pattern itself, any addition of high-frequency noise can 
potentially partially obscure the watermark and make it more 
difficult to detect. This is specifically the case if a derived 
copy is produces by scanning a printed copy of a marked image. 
Substantial high-frequency noise is added to a marked image by 
the screening process used in preparation for its printing. 
Printing ordinarily is accomplished with one or several inks or 
dyes that each have an invariable color. The screening process 
produces various shades of the invariably colored inks or dyes, 
needed to reproduce the color components, by covering the spatial 
area represented by each pixel with a finer grids of dots of the 
inks or dyes. Each of the grids of dots has a varying spatial 
density of the inks or dyes, and each dot in a grid of dots is 
significantly smaller spatially than the pixel area. The grids of 
dots so produced, one for each color component, spatial-ly replace 
the pixel they represent in the printed image copy, and, after 
fusion by the human viewing system, collectively produce a 
perceived correct color of the pixel. 

26 The screening process, by converting the components of each pixel 

27 into grids of dots of still smaller dimensions, inherently adds 

28 high-frequency artifacts and noise to the printed image copy that 
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1 were not in the original image. This can be verified easily by 

2 viewing a printed image under moderate magnification. If the 

3 printed image is then scanned to produce a derivative digitized 

4 image, the added high-frequency noise reproduced in the 

5 derivative copy is detrimental to watermark detection. It is to 

6 reduce the detrimental effects of the added high-frequency that a 

7 blurring filter is used. As stated above in the subsection titled 

8 "The Property of Explicit Low Frequency Content" the watermarking 

9 plane is designed to have significant low frequency content and 
1^1 will thus be relatively immune to the action of a blurring 

143 filter, but the high-frequency content of image, and more 

12j importantly the added noise, will be substantially attenuated. 

lpi An example rudimentary blurring filter can be implemented in the 

1© following manner. Each color plane of the image, represented as a 

l|p rectangular array of like color components, is partitioned, right 

ljP to left and top to bottom, into small sub-arrays that are three 

Iff] pixels high and three pixels wide. If the number of pixels in a 

l&jj row or column of the image array is not evenly divisible by 

19 three, the edge sub-arrays at the right or bottom of the 

20 partitioned image will contain fewer than nine pixels. The color 

21 components of the nine pixels in each sub-array (or fewer than 

22 nine if the sub-array is an edge sub-array) are averaged. The 

23 average value of the color components in each sub-array is then 

24 used to replace all the values in that sub-array. This completes 

25 the two-dimensional blurring filter. Those skilled in the art 

26 will recognize that there many other more sophisticated ways to 

27 implement a two-dimensional blurring filter. Nevertheless, in the 

28 method of the present invention the important desired result of 

29 applying any blurring filter remains the same as the that of 

30 applying the rudimentary filter described here, namely, the 
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1 reduction of high spatial frequencies and the preservation of low 

2 spatial frequencies of features in the derivative image. 

3 In the example rudimentary blurring filter presented, the first 

4 step of the method was to divide a marked image's color plane 

5 into nine element square sub-arrays. The choice of the size of 

6 the sub-arrays determines the degree to which high spatial 

7 frequencies among the pixel components in the marked image are 

8 reduced in the filtered image, if the marked image contains such 

9 high spatial frequencies, which it may not. By using nine element 
10, sub-arrays, the highest spatial frequency that can possibly exist 

in the filtered image is reduced by a factor of three. The larger 

the sub-array is chosen, the greater is the reduction of the 
W 

ip& highest possible spatial frequency that can exist in the filtered 
W image. It will be apparent to those skilled in the art that, when 
gj) applying a blurring filter, the degree to which the highest 
jfr. spatial frequency is to be reduced depends upon the degree to 
which high-frequency content in the watermarking plane used to 
produce the marked image was reduced. If the objective of using 
the blurring filter it to improve watermark detection, it would 

Mi 

20 become counter productive to reduce the high-frequency content of 

21 the marked image by a factor greater than that used in creating • 

22 the watermarking plane; to do so would remove not only 

23 undesirable high-frequency noise in the marked image but also 

24 some of the information contained in the imparted watermark. 
25 

26 Referring to Figure 16, a highly enlarged segment of an example 

27 watermarked image is shown. The watermark was imparted according 

28 to the method described previously herein. The modulation 

29 strength, P, used for the marking was 2.5 percent and visibility 
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1 of the watermark, even at high magnification is classified as 

2 undetectable invisible. Referring to Figure 17, a similarly 

3 enlarged segment of a derivative image is shown; it is derived 

4 from the marked image , shown in Figure 16 after it is screened in 

5 preparation for printing, forming a screened image, and 

6 subsequently printed and scanned. Note that significant 

7 high-frequency noise resulting from the screening process is 

8 evident in Figure 17. Figure 18 shows a filtered image produced 

9 by applying the rudimentary blurring filter to the derived image. 
Ig-j The noise reduction resulting from application of the blurring 
1'lQ filter is evident by comparing Figure 18, after the application, 
l|j with Figure 17, before the application. 

if 

llU, Watermark detection was attempted for each of the three images, 
the enlarged segments of which are shown in Figures 16, 17, and 

1|3 18. The watermark visualizer-coincidence images realized form the 
detection using the original marked image is shown in Figure 19. 

H 

17j1 The detection is a perfect detection. The original marked image 

lQ is then screened for printing, printed and scanned to form the 

19 derivative image. The watermark visualizer-coincidence image 

20 realized form the detection using the derivative image is shown 

21 in Figure 20. The detection is very weak, nearly nonexistent. 

22 After the rudimentary blurring filter is applied to the 

23 derivative image to form the filtered image, watermark detection 

24 is again attempted. The watermark visualizer-coincidence image 

25 realized form the detection using the filtered image is shown in 

26 Figure 21. The detection, although imperfect, is very strong, 

27 testifying to the efficacy of the use of the blurring filter 

28 before attempting watermark detection. 
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1 Use of a blurring filter is advantageous before any attempted 

2 watermark detection, regardless of the robust watermarking method 

3 used. If a watermark is robust, that is, if it is resistant to 

4 attacks, it must ordinarily contain significant low-frequency 

5 content. The low-frequency content of the watermark will not be 

6 unduly disturbed by the blurring filter, since the blurring 

7 filter is by its nature a low-pass filter. Any 

8 detection-disturbing high-frequency content in the image, whether 

9 occurring naturally as a part of the image or whether added by 
10 artificial means, such as screening in preparation for printing, 
lffl will be suppressed by the action of the blurring filter. The 

actual amount of blurring is generally dependent upon the 

lfel particular application and/or watermark. This is determined in 

Ijftj ways known to those skilled in the art. 

La 

15 Although the description is made for particular embodiments, 

lSI techniques and arrangements, the intent and concept of the 

1£ present invention are suitable to other embodiments, techniques 

and arrangements. For example, an obvious choice, and the choice 

lj^ of last resort, in demonstrating the existence of a watermark in 

20 a manipulated marked image is to again impart the watermark onto 

21 a copy of the unmarked original digitized image, and to use the 

22 color planes of that reconstituted marked image as ideal 

23 substitutes for the watermarking plane. The disadvantage of this 

24 alternative method is that it requires access to a copy of the 

25 unmarked original image. The visualizer can also have multiple 

26 color planes. The visualizer can be employed without the selector 

27 by having at least one statistical value associated with each 

28 pixel of the visualizer. Also, sequential repositioning of the 

29 selector on the reconstituted watermarking plane need not be 

30 non-overlapping. Non-overlapping selector positions in the 
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1 presented embodiment represent only a computational 

2 simplification. Also, a small random but coherent image may be 

3 included in the watermarking plane at positions known only to the 

4 marker and/or marking entity; if the so constituted watermarking 

5 plane were imparted onto a uniform color plane with strong 

6 modulation strength, the coherent image would be visible without 

7 use of a visualizer. Other methods of watermark detection and/or 

8 demonstration may be employed. These may for instance utilize any 

9 of the many statistical relationships between elements and their 
10 neighbors or non-neighbors. The robust techniques presented here 
lS3 may be used in combination with visible watermarking techniques 
12jFj as well as fragile invisible techniques. It will be clear to 

those skilled in the art that other modifications to the 

H 

ljf|j disclosed embodiments can be effected without departing from the 

1^ spirit and scope of the invention. 

The present invention can also be realized in embodiments of an 

lj&* apparatus having mechanisms for implementing the methods of the 

present invention as described herein in manners known to those 

lfe skilled in the art. For example, the present invention can also 

20 be realized as an apparatus to impart a watermark onto a 

21 digitized image, said apparatus comprising: means for providing a 

22 digitized image having at least one image plane, said image plane 

23 being represented by an image array having a plurality of pixels, 

24 said pixel having at least one color component, said watermark 

25 being formed using a distinct watermarking plane represented by 

26 an array having a plurality of distinct watermarking elements, 

27 each of said distinct watermarking elements having an array 

28 position and having one-to-one positional correspondence with 

29 said image pixels; and means for multiplying said brightness data 

30 associated with said at least one color component by a 
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1 predetermined brightness multiplying factor, wherein said 

2 brightness multiplying factor is a corresponding distinct 

3 watermarking element, said distinct watermarking element being in 

4 the domain of 0.5 to 1.0. Thus in an embodiment the present 

5 invention can also be realized as an apparatus for imparting a 

6 watermark onto a digitized image comprising the steps of: means 

7 for providing said digitized image comprised of a plurality of 

8 pixels, wherein each of said pixels includes brightness data that 

9 represents a brightness of at least one color; and means for 
} multiplying said brightness data associated with at least one of 
{ said pixels by a predetermined brightness multiplying factor in 
I the domain of 0.5 to 1.0. In a particular embodiment of the 

apparatus the image has I rows and J columns, and has a pixel in 

IP row i and column j having a brightness Y(i,j), and the means for 

multiplying includes: means for adding to or subtracting from the 

l&jj brightness Y(i,j) a different small random value e(i,j), wherein 

* 1 < i < I and 1 < j < J are the row and column indices of a pixel 

i location in the image. 

19 Thus in an embodiment the present invention can also be realized 

20 as an apparatus for imparting a watermark onto a digitized image 

21 comprising: means for providing said digitized image comprised of 

22 a plurality of pixels, wherein each of said pixels includes 

23 brightness data that represents a brightness of at least one 

24 color, with said image having I rows and J columns, and a pixel 

25 in row i and column j having a brightness Y(i,j); and means for 

26 adding to or subtracting from the brightness Y(i,j), for all i 

27 and all j, a random value wherein 1 < i < I and 1 < j < J 

28 are the row and column indices of a pixel location in the image. 
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1 Thus in an embodiment the present invention can also be realized 

2 as an apparatus for generating a watermarked image, the apparatus 

3 comprising: means for imparting a watermark onto a digitized 

4 image having a plurality of original pixels, each of said pixels 

5 having original brightness values; means for providing said 

6 digitized watermarking plane comprising a plurality of 

7 watermarking elements, each. element having a watermark brightness 

8 multiplying factor and having one-to-one positional 

9 correspondence with said original, pixels; and means for producing 
10 a watermarked image by multiplying said original brightness 

ljj values of each of said original pixels by said brightness 

12p multiplying factor of a corresponding one of said watermark 

w 

elements. 

Thus in an embodiment the present invention can also be realized 

lL as an apparatus comprising: means for forming a watermarking 

165 plane including a plurality of elements each having a brightness 

lpC adding or subtracting value; means for generating a robust random 

fe * 

1© sequence of integers having a first plurality of bits; means for 
y, 

19 linearly remapping said random sequence to form a remapped 

20 sequence of brightness multiplying factors to provide a desired 

21 modulation strength; means for computing a discrete Fourier 

22 transform of said remapped sequence to form a Fourier sequence 

23 having frequency coordinates; means for expanding said frequency 

24 . coordinates to form an expanded sequence; means for computing an • 

25 inverse Fourier transform of said expanded sequence to obtain a 

26 watermarking sequence of values; and means for deriving said 

27 brightness adding or subtracting values of said elements of said 

28 watermarking plane based upon said watermarking sequence of 

29 values. 
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Thus in an embodiment the present invention can also be realized 
as an apparatus for detecting a watermarking plane comprising: 
means for providing an image having a plurality of pixels marked 
by the watermarking plane, said watermarking plane having a 
plurality of watermarking elements; means for aligning said 
watermarking plane with said image; means for identifying a 
subset of said image pixels; means for each image pixel, u(i,j), 
wherein 1 < i < I and 1 < j < J, of said subset of image pixels, 
including means for generating a first value representing a 
relationship between an attribute of said image pixel u(i,j) and 
an attribute of image pixels that neighbor said image pixel 
u(i,j); means for identifying a watermarking element w(i,j) that 
positionally corresponds to said image pixel u(i,j) and 
watermarking elements that correspond to said image pixels that 
neighbor said image pixel u(i,j); means for generating a second 
value representing a relationship between an attribute of said 
watermarking element w(i,j) and an attribute of the identified 
neighboring watermarking elements; and means for generating a 
coincidence value representing likelihood that said image is 
marked by said watermarking plane based upon said first and 
second values. 

Thus in an embodiment the present invention can also be realized 
as an apparatus comprising means for generating a visual 
representation of a data array of data elements having a data 
array size, including: means for providing a 

visualizer-coincidence pattern of visualizer-coincidence image 
pixels represented by a visualizer-coincidence array of 
visualizer-coincidence pixels, .said visualizer-coincidence array 
having an array size equal to said data array size, wherein each 
of said visualizer-coincidence pixels has a first color if a 
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1 corresponding data element is a first logical value and a second 

2 color if said corresponding data element has a complementary 

3 logical value; means for setting said visualizer-coincidence 

4 pixel to a first color if a value of said data element is above a 

5 predetermined threshold and to another color if said value is 

6 below said predetermined threshold; and means for displaying said 

7 visualizer-coincidence image to form said visual representation. 

8 Thus in an embodiment the present invention can also be realized 

9 as an apparatus for imparting a watermark onto a digitized image 
lj&j comprising: means for providing said digitized image comprised of 
l4j a plurality of pixels, wherein each of said pixels includes 

lgj brightness data represented by at least one color component, Y; 

lEf and means for adding to or subtracting from said brightness data 

associated with at least one of said pixels a predetermined 

V$ brightness adding or subtracting factor in the range of -0.5Y to 

+0.5Y, wherein said brightness adding or subtracting factor has a 

1^ relationship with a number taken from a random number sequence, 

1§1 said relationship is a linear remapping to provide a desired 

lp modulation strength, and said modulation strength is less than 50 

20 percent. 

21 Thus in an embodiment the present invention can also be realized 

22 as an apparatus for imparting a watermark onto a digitized image 

23 comprising: means for providing said digitized image comprised of 

24 a plurality of pixels, wherein each of said pixels includes 

25 brightness data represented by at least one color component, Y; 

26 and means for adding to or subtracting from said brightness data 

27 associated with at least one of said pixels by a predetermined 

28 brightness adding or subtracting factor in the range of -0.5Y to 

29 +0.5Y, wherein said brightness adding or subtracting factor has a 
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1 relationship with a number taken from a random number sequence, 

2 said relationship is a linear remapping to provide a desired 

3 modulation strength, said sequence is formed from a plurality of 

4 robust watermarking parameters, and said parameters comprise a 

5 cryptographic key, two coefficients and an initial value of said 

6 random number generator. 

7 The present invention can be realized in hardware, software, or a 

8 combination of hardware and software. A visualization tool 

9 according to the present invention can be realized in a 

lffl centralized fashion in one computer system, or in a distributed 

l|j fashion where different elements are spread across several 

li&J interconnected computer systems. Any kind of computer system - or 

l^j other apparatus adapted for carrying out the methods and/or 

14* functions described herein - is suitable. A typical combination 

£3 

15 of hardware and software could be a general purpose computer 

IP system with a computer program that, upon being loaded and 
03 

Ipj, executed, controls the computer system such that it carries out 
the methods described herein. The present invention can also be 

Q 

1S& embedded in a computer program product, which comprises all the 

20 features enabling the implementation of the methods described 

21 herein, and which - when loaded in a computer system - is able to 

22 carry out these methods. 

23 Computer program means or computer program in the present context 

24 include any expression, in any language, code or notation, of a 

25 set of instructions intended to cause a system having an 

26 information processing capability to perform a particular 

27 function either directly or after either or both of the 

28 following: conversion to another language, code or notation, 

29 and/or reproduction in a different material form. 
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1 Thus the invention includes an article of manufacture comprising 

2 a computer usable medium having computer readable program code 

3 means embodied therein for causing a function described above, 

4 The computer readable program code means in the article of 

5 manufacture comprises computer readable program code means for 

6 causing a computer to effect the steps of a method of this 

7 invention. Similarly, the present invention may be implemented as 

8 a computer program product comprising a computer usable medium 
|g having computer readable program code means embodied therein for 
^ causing a function described above. The computer readable program 
|y code means in the computer program product comprises computer 

2^ readable program code means for causing a computer to effect one 

or more functions of this invention. Furthermore, the present 

jF invention may be implemented as a program storage device readable 

S3 by machine, tangibly embodying a program of instructions 

jjjjj executable by the machine to perform method steps for causing one 

?H or more functions of this invention. 

18 It is noted that the foregoing has outlined some of the more 

19 pertinent objects and embodiments of the present invention. This 

20 invention may be used for many applications. Thus, although the 

21 description is made for particular arrangements and methods, the 

22 intent and concept of the invention is suitable and applicable to 

23 other arrangements and applications. It will be clear to those 

24 skilled in the art that modifications to the disclosed 

25 embodiments can be effected without departing from the spirit and 

26 scope of the invention. The described embodiments ought to be 

27 construed to be merely illustrative of some of the more prominent 

28 features and applications of the invention. Other beneficial 

29 results can be realized by applying the disclosed invention in a 
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1 different manner or modifying the invention in ways known to 

2 those familiar with the art. 



a 
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